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In the summer of 1996, the U, S, Armed Services will 
implement the computerized adaptive testing version of the Armed 
Services Vocational Aptitude Battery (CAT-ASVAB) . When conversion is 
completed at the 65 targeted military entrance processing stations, 
about half the applicants will take this test version. This paper 
describes the efforts that led to the decision to implement the 
CAT-ASVAB, progress in its implementation, and the next steps in 
research on improvements in Department of Defense testing programs. 
The adaptive nature of the new format significantly reduces the time 
required to determine aptitudes of examinees as well as time to 
obtain official scores of record. Estimation of potential cost 
savings led to* Lhe decision to impjemen*" the computerized version. 
Hardware and software requirements were determined, and the adaptive 
version of the test, which allows items to be presented to the 
applicant based on his or her success in answering previous 
questions, was developed. If implementation progresses as expected, 
use of the computerized test will be expanded to other military 
evaluation sites, (Contains two references,) (SLD) 
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Implementation of the Computerized Adaptive Version of the Armed Services 

Vocational Aptitude Battery' 

Linda T. Curran 
Linda A. Jordan 

Defense Manpower Data Center 

In the Summer of 1996, aptitude testing for enlistment in the Armed Services of 
the United States is going high-tech with the implementation of the computerized adaptive 
testing version of the Armed Services Vocational Aptitude Battery (CAT-ASVAB). 

When the conversion is complete at the 65 Military Entrance Processing Stations (MBPS), 
approximately half of all applicants will be tested with the CAT-ASVAB. CAT-ASVAB 
provides numerous advantages over the current paper-and-pencil (P&P) lock-step 
administration of the battery, including shorter testing time, increased test security, 
decreased test development costs, flexible start times, and immediate availability of scores. 
This paper will describe the efforts that led to the decision to implement CAT-ASVAB, 
the progress in implementation at the MBPS, and next steps in research on improvements 
in the Department of Defense (DoD) testing programs. 

Background 

ASVAB. The ASVAB is a multiple aptitude battery used by the Military Services 
to determine the enlistment eligibility of applicants and to assign qualified applicants to 
military occupational specialties. The ASVAB is administered at 65 MBPS, where other 
aspects of enlistment processing also are conducted, and at about 700 Mobile Examining 
Team (MET) sites, located nearer to the applicant, where only ASVAB testing is 
conducted. Over 600,000 tests are administered annually in MBPS and MET sites. 

P&P and computer adaptive versions of the ASVAB are available for 
administration at the enlistment testing locations. The primary mode of ASVAB 
administration is lock-step as a P&P test. In P&P mode, the ASVAB requires about three 
and one-half hours to administer in P&P mode and is scored by an optical mark reader 
(OMR). The score sheets from tests administered at MET sites must be physically 
transported to MBPS for official scoring, delaying the receipt of official scores by one to 
three or more days. 

The CAT-ASVAB is adaptive in that items presented to the applicant depend on 
the applicant’s success in answering previous questions. The adaptive nature of the test 
significantly reduces the time required to determine aptitudes of examinees and time to 
obtain official scores of record. On average, examinees can complete the CAT-ASVAB in 
less than two hours. This reduction in testing time, in comparison with the P&P test, 
combined with reduced costs for computers has made CAT-ASVAB a viable alternative to 
P&P testing. At this time, the CAT-ASVAB is administered in five of the 65 MBPS and 
at one MET site location as part of an operational test and evaluation (OT&E). 

' Presented at the National Council on Measurement in Education, New York City, April 1996 
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Review of the Enlistment Testing Process. About five years ago, the ASVAB 
Review effort was initiated to review the total enlistment processing system with an 
emphasis on aptitude testing, namely the administration, content, and scoring of the 
ASVAB in the ETP. The purpose of the ASVAB Review was to develop and evaluate 
various ASVAB alternatives and to make recommendations based on cost, technical, and 
policy considerations. 

The initial step to identify alternative testing concepts was to review current 
aptitude processing and identify strengths and weaknesses. An analysis of the current 
aptitude processing, presented in detail in Hogan and Mullin (1993), found the system to 
operate reasonably well at relatively low capital costs plus testing capacity can be 
expanded rapidly with minimal expenditure. Hogan and Mullin identified a few areas for 
potential improvement. One of these is the length of the P&P battery, which at three and 
one-half hours may prevent some prospects from applying, does reduce the efficiency of 
recruiters who wait while their applicants take the test, and causes most MEPS processing 
to take two days or more. A second area is the lag time between test completion and 
availability of official scores; this further reduces recruiter efficiency, and may cause some 
applicants to lose their motivation to enlist. The third area is the rigidity in processing. 

The P&P ASVAB is administered lock-step with separately timed tests. Applicants who 
arrive late to a test session (after testing has begun) are not allowed to test and must be 
rescheduled; the time of the applicant and the recruiter who brought the applicant to 
testing and travel costs are wasted in this situation. 

Although a large number of operational changes were initially considered, the list 
was reduced to eight options that included various combinations of two kinds of changes: 
1) automation of some or all of the aspects of test administration and scoring through the 
use of computers or digital response testing (DRT) pads and 2) expansion of MET site 
concepts to include ASVAB administration in commercial testing centers (CTCs) staffed 
by contractor personnel. DRT pads are hand-held touchpads the size of large calculators, 
are used by examinees to record their answers, and can be hooked up to modems to 
transmit raw item responses to a central location for scoring. Contract test centers are 
commercial facilities that administer primarily computer-based tests. Over the past few 
years, the number of commercial testing facilities has grown and some companies have 
hundreds of testing sites available throughout the country. For more information on the 
process to identify alternative testing concepts see McBride (1993). 

All of the concepts involving computerized testing in the MEPS resulted in net 
savings. Table 1 presents savings and costs associated with implementing CAT-ASVAB 
at the MEPS. Experience with the OT&E of CAT-ASVAB in the MEPS convinced 
policymakers that the concepts being evaluated were, in fact, feasible, and that the results 
reasonably represented likely costs and benefits. On 13 May 1993, the Manpower 
Accession Policy Steering Committee (MAPSC) approved plans to implement CAT- 
ASVAB in all MEPS as soon as possible and to continue research on automating testing at 
MET sites. 
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Table 1: Key Costs and Savings from Implementation of CAT-ASVAB at MEPS 



Cost and Savings Categories Amounts 

Estimated Annual Costs of Implementing: (Equipment, Training, Site Mods)*$ 660,000 



Estimated Annual Savings 

From Reduced Meals and Lodging: $2,000,000 

From Reduced Administration Costs: (includes scanning) $ 300,000 

From Reduced Printing and Scoring Costs: $ 400,000 

From Increased Recruiter Efficiency: $1,000,000 

From Reduced Test Development Costs: $ 50.000 



ANNUAL TOTAL FOR KEY SAVINGS: $3,750,000 

Estimated Annual Net Savings: $3,090,000 



*Note: assumes a five-year life cycle for computer hardware. 



Progress in Implementation 

Since the MAPSC decision to implement CAT-ASVAB in the MEPS, notable 
progress has been made in planning and executing implementation steps. The steps fall 
under the general categories of 1) hardware purchase and set-up, 2) equating of new 
hardware, and 3) administration plan. Each will be described in turn along with 
corresponding progress. 

Hardware Purchase and Set-up. To determine the amount of money that would 
be required and to initiate the purchase of the computers, the first step in the 
implementation planning process was to address equipment specifications and quantity 
requirements. The computer specifications were defined by memory requirements of the 
adaptive testing system, the desire to use the computers in a networking mode within a 
MEPS, plus the requirement to have the CAT-ASVAB testing system interface with the 
information management system in use at the MEPS. Minimum requirements for desktop 
computers for the MEPS include the following specifications: 80486 (Intel or Intel 
Compatible) microprocessor with 25 Mhz, 4MB RAM, VGA color monitor with 14 inch 
CRT, extended graphics resolution modes (640 X 480 pixels), .28 mm dot-pitch, and 
video adapter (512 Kb Video RAM); and 3.5" 1 .44 Mb high density floppy disk drive and 
40 Mb internal hard disk drive. 

The numbers of computers needed at each MEPS were determined based on P&P 
testing session sizes. The distribution of testing sizes for each MEPS for a fiscal year were 
analyzed, and the testing session sizes at the 75th percentile was used as a conservative 
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estimate of the required number of computers. On average, about 20 computers per 
MBPS are required, for a total of around 1,400. 

Besides hardware and software costs, costs associated with equipment 
maintenance and the upgrading of MBPS facilities to accommodate a computerized testing 
platform needed to be considered. To address the facilities issue, the Military Bntrance 
Processing Command conducted a site survey of the MBPS. Results of the survey 
indicated that electrical upgrades and the purchase of tables and chairs would be required 
at the sites. In all, costs for the equipment, maintenance of the equipment, furniture, and 
upgrades of testing facilities will be about $3.3 million. 

One of the most critical steps in the implementation was acquiring the funding to 
pay for all the costs associated with fielding CAT-ASVAB. A small amount of funds were 
expended in FY 1995 to begin the electrical upgrades of the MBPS. The bulk of the funds 
were acquired this year. Plans are to conduct a phased implementation of CAT-ASVAB 
over a six month time period beginning this fall. In the first phase, computers will be 
implemented at 20 MBPS locations. Four teams of staff members from the Defense 
Manpower Data Center will set-up the computers and provide training to MBPS personnel 
on computer set-up and administration of the CAT-ASVAB. 

Equating of New Hardware. In a hardware effects study conducted by the Navy 
Personnel Research and Development Center (NPRDC), it was observed that some of the 
tests on the CAT-ASVAB (the speeded tests, in particular) were sensitive to differences in 
computer hardware. Specifically, scores on Numerical Operations and Coding Speed 
when using desktop computers differed significantly from scores obtained using 
notebooks. Since new and better equipment will be purchased for implementation, new 
conversion tables based on the equating of new hardware to the P&P ASVAB (the 
reference version) must be accomplished. Current conversion tables that equate CAT- 
ASVAB to P&P ASVAB scores are based on use of Hewlett-Packard machines and 
therefore can’t be used for the new equipment. NPRDC is in the process of conducting an 
equating on desktops, similar to what will be used in the MBPS, and notebooks, which 
potentially will be used at MBT sites. The data collection is complete and data analyses 
and development of new conversion tables are underway. 

Administration Plan. With the implementation of CAT-ASVAB, administration 
conditions (test instructions and timing) will be completely standardized. Administration 
will also be completely individualized so examinees can start whenever a machine is 
available and move at their own pace throughout the examination. Because of the 
adaptive nature of the test, it will be more engaging, with a minimum of questions that are 
too difficult or too easy. Scores will be immediately available and will be free from errors 
associated with marking and scanning answer sheets. Printed test booklets will be 
eliminated and test security will be correspondingly increased. 

experience with CAT-ASVAB has shown that the most preferable option for 
CAT-ASVAB administration includes a flexible start window. CAT-ASVAB 
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administration in the OT&E involves a two hour window in which examinees arrive for 
testing. This is an advantage over P&P testing which is lock-step in its administration; 
examinees who do not arrive in time to begin a P&P test are turned away. With a two 
hour window in a four hour testing session (approximately the time allowed for P&P 
testing), two seatings of the CAT-ASVAB test can be accomplished, thereby enabling the 
testing session to accommodate twice as many applicants in a day. 

In the CAT-ASVAB OT&E, the computers examinees use to take CAT-ASVAB 
are set up in a network with a test administrator (TA) station. The software is designed to 
operate in stand-alone mode if the network fails. The network mode enables the TA to 
randomly assign examinees to test stations, track the progress of examinees, and 
download and print test scores. Immediate feedback of test scores is provided to 
applicants and recruiters after testing. This is a distinct advantage over the P&P ASVAB 
where answer sheets are scanned and scored overnight and scores are provided to the 
applicant and recruiter the day after testing. The same administration arrangement, as in 
the CAT-ASVAB OT&E, will be continued after implementation with the new hardware. 

Next Steps 

Further research is continuing in the development and evaluation of alternatives to the 
current method of enlistment testing, paper-and-pencil administration, at the MET sites. 
The MET site alternatives to paper and pencil testing are similar to the ones used for 
implementation at the MEPS: CAT-ASVAB and DRTs. CAT-ASVAB could be 
administered at MET sites using notebook or desktop computers, or at CTCs. Another 
option is to remain with paper-and-pencil administration. 

Implementation of automated ASVAB testing in the MET sites raises issues not 
associated with implementation at the MEPS, such as storage and set-up of computers, 
transportation of equipment, availability of an adequate electrical supply, and availability 
of a telephone for data communications. While MEPS are permanently established, MET 
sites generally are not; rather a site may be a room in a National Guard Armory, a Post 
Office, or even a hotel, for example. Furthermore, MET sites differ substantially among 
themselves; thus, a reasonable option for one site may be unworkable at another. 

There are two phases in the project to develop and evaluate alternative MET site 
concepts. The first phase is to conduct feasibility studies of various alternatives. The 
MET sites, unlike the MEPS, vary in terms of their operating hours and facilities. Some 
MET sites have high flow rates and are open once a week. Other MET sites do not test as 
many people and are open once or twice a month. To learn more about whether 
computerized or digital response testing would be feasible at MET sites, the MEPCOM 
conducted a site survey of the MET sites to determine each MET site’s accommodations 
and hours of operation. In terms of facilities, the MET sites vary significantly in the 
availability of telephone lines, number and type of electrical outlets, accessibility (distance 
of testing room from parking, need to walk up stairs, etc.), type of furniture, availability of 
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storage, etc. Also, the number of MET sites can vary from week to week, MET site 
locations change frequently, and the flow rates through the sites are not as predictable. 

To develop accurate estimates of costs associated with various MET site alternatives, 
the second phase of the effort will try out the various alternatives and will provide an 
empirical basis upon which to estimate the costs. Although a particular concept may be 
feasible, it may not be cost effective when implemented. Therefore, a pilot test of 
computerized and digital response testing and CTCs is necessary to accurately evaluate 
alternative concepts. As there are about 700 MET sites, it is not feasible to try out 
various testing alternatives at all of the sites. Therefore, a representative sample of MET 
sites will be selected in which to conduct the studies. This test will determine the effect of 
the new testing MET site alternative on travel costs, recruiter behaviors in taking the 
applicants to various testing locations, and applicant behaviors in willingness to take the 
ASVAB, or on possible restructuring of the number and locations of MET sites. 
Ultimately, the cost benefit evaluation of the MET site alternatives will enable DoD 
policymakers to weigh the advantages and disadvantages of various test administration 
media against associated costs to make a decision as to which alternative or combination 
of alternatives will be used in future MET site testing operations. This cost benefit 
evaluation is scheduled to be completed early in 1997. 
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